Papers with information retrieval
Human-in-the-loop Schema Induction (2023.acl-demo)
Copied to clipboard
Tianyi Zhang, Isaac Tham, Zhaoyi Hou, Jiaxuan Ren, Leon Zhou, Hainiu Xu, Li Zhang, Lara Martin, Rotem Dror, Sha Li, Heng Ji, Martha Palmer, Susan Windisch Brown, Reece Suchocki, Chris Callison-Burch
| Challenge: | Existing approaches to event-centric natural language understanding (NLU) have been limited to linear and temporal ones. |
| Approach: | They propose a human-in-the-loop schema induction system powered by GPT-3 . they show that it transfers to new domains more easily than previous approaches . |
| Outcome: | The proposed system transfers to new domains more easily than previous approaches and reduces human curation. |
UKP-SQUARE: An Online Platform for Question Answering Research (2022.acl-demo)
Copied to clipboard
Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Gregor Geigle, Max Eichler, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Şahin, Iryna Gurevych
| Challenge: | Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats and require different model architectures and setups. |
| Approach: | They propose an extensible online QA platform that allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
| Outcome: | The proposed tool allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
Wikipedia as a Resource for Text Analysis and Retrieval (P19-4)
Copied to clipboard
| Challenge: | Tutorial examines the role of Wikipedia in tasks related to text analysis and retrieval. |
| Approach: | tutorial examines the role of Wikipedia in tasks related to text analysis and retrieval. |
| Outcome: | This tutorial examines the role of Wikipedia in tasks related to text analysis and retrieval. |
AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysis (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Recent advances in AI focus on multi-agent systems (MAS) that can be integrated with Large Language Models (LLMs) but current systems still face challenges of inter-agency communication, coordination, and interaction with heterogeneous tools and resources. |
| Approach: | They propose a modular multi-protocol MAS framework with self-implemented A2A and MCP . the framework supports natural language interaction without prior technical expertise . |
| Outcome: | The proposed framework supports natural language interaction without prior technical expertise and responds to multimodal queries for tasks including information retrieval, question answering, and image analysis. |
NoEl: An Annotated Corpus for Noun Ellipsis in English (2020.lrec-1)
Copied to clipboard
| Challenge: | Ellipsis resolution is an important step to improve the accuracy of mainstream natural language processing tasks such as information retrieval, event extraction, dialog systems, etc. |
| Approach: | They extend the study of ellipsis by annotating a corpus for noun ellippsis and closely related phenomenon using the first hundred movies of Cornell Movie Dialogs Dataset. |
| Outcome: | The proposed corpus has 946 instances of exophoric and endophorical noun ellipsis, making it the biggest resource of nouns in English, to the best of our knowledge. |
Graph-augmented Learning to Rank for Querying Large-scale Knowledge Graph (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing knowledge graph question answering methods only search for the answer in a large knowledge graph. |
| Approach: | They propose to partition retrieved knowledge subgraphs into smaller sub-KSGs and then use a graph-augmented learning to rank method to select the top-ranked sub-kSGs. |
| Outcome: | The proposed method can capture global interactions in question and subgraphs and local interactions on the full KSG and top-ranked sub-KSGs respectively. |
BeefBot: Harnessing Advanced LLM and RAG Techniques for Providing Scientific and Technology Solutions to Beef Producers (2025.coling-demos)
Copied to clipboard
| Challenge: | Generic Large Language Models (LLMs) are useful for information retrieval but often hallucinate and fail to deliver tailored solutions to the specific needs of beef producers. |
| Approach: | They propose to use Retrieval-Augmented Generation and fine-tuning to build a chatbot for beef producers that retrieves latest agricultural technologies and scientific insights. |
| Outcome: | The proposed chatbot retrieves latest agricultural technologies, practices and scientific insights to provide rapid, domain-specific advice. |
AttriSage: Product Attribute Value Extraction Using Graph Neural Networks (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods for extracting attribute value from product descriptions are limited in their accuracy. |
| Approach: | They propose a method for extracting product attribute value from product description using graphs and neural networks. |
| Outcome: | The proposed method improves product description attribute value extraction accuracy compared to baseline methods. |
BiomedCurator: Data Curation for Biomedical Literature (2022.aacl-demo)
Copied to clipboard
Mohammad Golam Sohrab, Khoa N.A. Duong, Ikeda Masami, Goran Topić, Yayoi Natsume-Kitatani, Masakata Kuroda, Mari Nogami Itoh, Hiroya Takamura
| Challenge: | BiomedCurator uses state-of-the-art natural language processing techniques to extract structured data from scientific articles. |
| Approach: | They propose a web application that extracts structured data from PubMed and ClinicalTrials.gov . the application uses a combination of natural language processing techniques and a pattern-based extraction approach . |
| Outcome: | The proposed system extracts the structured data from PubMed and ClinicalTrials.gov datasets. |
Aligning LLMs for Multilingual Consistency in Enterprise Applications (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) remain unreliable for global enterprise applications due to performance gaps between high-resource and mid/low-resourced languages . |
| Approach: | They propose a batch-wise alignment strategy that aligns model outputs across languages . this method improves non-English accuracy by up to 23.9% without compromising English performance . |
| Outcome: | The proposed approach improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. |
FVQA 2.0: Introducing Adversarial Samples into Fact-based Visual Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Fact-based Visual Question Answering (FVQA) is a visual question answering task that requires information retrieval using common sense knowledge graphs to answer. |
| Approach: | They propose a new test question with adversarial variants to address this imbalance by using a KB-VQA dataset that is small and contains only one answer per question. |
| Outcome: | The proposed version reduces the vulnerability of the original FVQA dataset without human annotations. |
The Russian-focused embedders’ exploration: ruMTEB benchmark and Russian embedding model design (2025.naacl-long)
Copied to clipboard
| Challenge: | Embedding models are used in tasks such as information retrieval and semantic textual similarity. |
| Approach: | They propose a new Russian-focused embedding model called ru-en-RoSBERTa and a benchmark for Russian language . they propose to use the roMTEB benchmark to assess Russian and multilingual models . |
| Outcome: | The proposed model achieves results that are on par with state-of-the-art models in Russian. |
Situation-Based Multiparticipant Chat Summarization: a Concept, an Exploration-Annotation Tool and an Example Collection (2021.acl-srw)
Copied to clipboard
| Challenge: | Currently, text chat does not offer navigation or full-featured search, although the high volumes of messages demand it. |
| Approach: | They propose a data annotation tool for situation-based summarization that can be used to extract messages from chat logs. |
| Outcome: | The proposed tool is the first to be developed for situation-based summarization. |
An Effective Post-training Embedding Binarization Approach for Fast Online Top-K Passage Matching (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing models that learn semantic representations of passages are prone to performance degradation . embedding binarization is a promising branch of model compression . |
| Approach: | They propose an embedding binarization approach that can be used to optimize for online inference. |
| Outcome: | The proposed model can perform query-passage matching acceleration. |
Mitigating Bias in Item Retrieval for Enhancing Exam Assembly in Vocational Education Services (2025.naacl-industry)
Copied to clipboard
| Challenge: | Despite the practical importance of exam assembly, few methods exist to support educators during manual item retrieval for exam assembly tasks. |
| Approach: | They propose a mixed-integer programming re-ranking approach to improve relevance while mitigating bias on an industry-grade exam assembly platform. |
| Outcome: | The proposed approach improves relevance and reduces bias by 17% when compared to other methods on a real-world exam assembly platform. |
CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search (2025.naacl-srw)
Copied to clipboard
Nikita Sorokin, Tikhonov Anton, Dmitry Abulkhanov, Ivan Sedykh, Irina Piontkovskaya, Valentin Malykh
| Challenge: | clone detection is crucial in software development for identifying semantically similar code . clones can be found in the same language code snippets, but there is little research on multilingual clonage detection. |
| Approach: | They propose a novel training procedure leveraging cross-lingual similarity to train language models on source code in various programming languages. |
| Outcome: | The proposed method achieves state-of-the-art on C++ and Python clone detection benchmarks with comparable performance on decoder-based models. |
ContrastiveMix: Overcoming Code-Mixing Dilemma in Cross-Lingual Transfer for Information Retrieval (2024.naacl-short)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have been widely adopted in cross-lingual transfer . however, training mPLMs on code-mixed data is counterproductive . |
| Approach: | They propose a method that disentangles contrastive loss between conflicting objectives . they propose mPLMs that train on code-mixed data for information retrieval . |
| Outcome: | The proposed model outperforms mDPR in cross-lingual transfer tasks . it is effective in sentence-level and token-level zero-shot tasks despite lack of target language data . |
Alignment Analysis of Sequential Segmentation of Lexicons to Improve Automatic Cognate Detection (P18-3)
Copied to clipboard
| Challenge: | Existing studies on cognate detection only distinguish between a pair of words whether they are cognates or non-cognates. |
| Approach: | They propose to incorporate ranking functions into search engine ranking functions . they also propose to use graphical error modelling to calculate morphological shifts . |
| Outcome: | The proposed methods give better results than competing baselines, the authors show . they show that language modelling based retrieval functions with positional tokenization and error modelling give better outcomes . |
ClinQueryAgent: A Conversational Agent for Population Health Management (2026.acl-demo)
Copied to clipboard
| Challenge: | In 2014 there were 2.3B SNOMED CT codes recorded in English healthcare practices. By 2024, this number had grown almost threefold to 6.1B codes: approximately 100 codes per person each year. |
| Approach: | They introduce a system for translating natural language population health questions into executable database queries using agents with access to both local and external knowledge bases. |
| Outcome: | The proposed system is able to handle a range of health informatics tasks on three datasets and via a beta-testing phase. |
CiteFix: Enhancing RAG Accuracy Through Post-Processing Citation Correction (2025.acl-industry)
Copied to clipboard
| Challenge: | Retrieval Augmented Generation (RAG) is a powerful application of Large Language Models (LLMs). |
| Approach: | They propose to use BERTScore to fine-tune Large Language Models on domain-specific data to improve citation accuracy. |
| Outcome: | The proposed approach improves citation accuracy by 15.46% with minimal latency and cost. |
Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking (2022.emnlp-main)
Copied to clipboard
| Challenge: | In information retrieval, candidate set pruning is used to speed up two-stage relevance ranking but lacks accurate error control and empirical guarantees. |
| Approach: | They propose a method that guarantees the test error after pruning is controlled under a user-specified threshold with high probability. |
| Outcome: | The proposed method reduces the average set size from 1000 to 27, increasing reranking speed by about 37 times while keeping MRR@10 greater than a pre-specified value of 0.38 with about 90% empirical coverage. |
OpenKE: An Open Toolkit for Knowledge Embedding (D18-2)
Copied to clipboard
| Challenge: | Existing knowledge embedding tools are available for embeddable knowledge graphs. |
| Approach: | They propose a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space. |
| Outcome: | The toolkit and pre-trained embeddings are available on http://openke.thunlp.org/. |
Fréchet Distance for Offline Evaluation of Information Retrieval Systems with Sparse Labels (2024.eacl-long)
Copied to clipboard
| Challenge: | Obtaining high-quality labeled data that accurately represents complexity of real-world scenarios can be expensive, time-consuming, or even impractical. |
| Approach: | They propose to use Fréchet Inception Distance to measure distance between judged items and retrieved results. |
| Outcome: | The proposed method improves on a MS MARCO dataset and TREC Deep Learning Tracks query sets. |
RDoC Task at BioNLP-OST 2019 (D19-57)
Copied to clipboard
| Challenge: | BioNLP-OST is an international competition organized to facilitate development and sharing of computational tasks of biomedical text mining and solutions to them. |
| Approach: | They propose a new mental health informatics task that is composed of two subtasks: information retrieval and sentence extraction. |
| Outcome: | The proposed task performed well on both tasks, but there are still challenges. |
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)
Copied to clipboard
| Challenge: | OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch . |
| Approach: | OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment. |
| Outcome: | OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment . |
SEAGLE: A Platform for Comparative Evaluation of Semantic Encoders for Information Retrieval (D19-3)
Copied to clipboard
| Challenge: | Existing semantic text encoding models are limited in coverage and few attempts to empirically compare them on IR tasks have been made. |
| Approach: | They propose to implement word embedding aggregators and pretrained semantic encoders and to allow for their comparative evaluation on arbitrary IR collections. |
| Outcome: | The proposed model can be exploited via an easy-to-use web interface and its modular backend (micro-service architecture) can easily be extended with additional semantic search models. |
Weakly-supervised Deep Cognate Detection Framework for Low-Resourced Languages Using Morphological Knowledge of Closely-Related Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to cognate detection focus on orthographic, phonetic or contextual models, which under-perform for most under-resourced languages. |
| Approach: | They propose a language-agnostic weakly-supervised deep cognate detection framework for under-resourced languages using morphological knowledge from closely related languages. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on cognate detection datasets across languages and can be extended to a wide range of languages from any language family. |
Extractive Entity-Centric Summarization as Sentence Selection using Bi-Encoders (2022.aacl-short)
Copied to clipboard
| Challenge: | Entity-centric summarization is a type of controllable summarizing that aims to produce a summary specific to a given target entity. |
| Approach: | They propose to recast a sentence selection task as a controllable summarization using a dataset supported by EntSUM. |
| Outcome: | The proposed framework outperforms the current state-of-the-art in the sentence selection task and outperformed the competitive entity-centric Lead 3 heuristic by 1.1 F1. |
Advancing Topic Segmentation and Outline Generation in Chinese Texts: The Paragraph-level Topic Representation, Corpus, and Benchmark (2024.lrec-main)
Copied to clipboard
| Challenge: | Compared with sentence-level topic structure, paragraph-level topics can grasp and understand the context of a document from a higher level. |
| Approach: | They propose a hierarchical paragraph-level topic structure representation with three layers to guide corpus construction. |
| Outcome: | The proposed method achieves the largest Chinese paragraph-level topic structure corpus, achieving high quality. |
Backtracing: Retrieving the Cause of the Query (2024.findings-eacl)
Copied to clipboard
| Challenge: | a number of online content portals allow users to ask questions to supplement their understanding. |
| Approach: | They propose a task of backtracing to retrieve the text segment that most likely caused a user query. |
| Outcome: | The proposed method improves on the backtracing task in three domains . the results show that there is room for improvement and new retrieval approaches . |
Towards Safer Operations: An Expert-involved Dataset of High-Pressure Gas Incidents for Preventing Future Failures (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing datasets for incident management tasks are labor-intensive and time-consuming. |
| Approach: | They propose a new IncidentAI dataset for safety prevention that includes three tasks . they argue that NLP techniques are beneficial for analyzing incident reports . |
| Outcome: | The proposed dataset shows that NLP techniques are beneficial for analyzing incident reports to prevent future failures. |
Vector Space Interpolation for Query Expansion (2022.aacl-short)
Copied to clipboard
| Challenge: | Topic-sensitive query set expansion is crucial for queries related to sensitive and emerging topics. |
| Approach: | They propose a method for topic-sensitive query set expansion using vector space interpolation. |
| Outcome: | The proposed method generates new queries about the sensitive topic by incorporating set diversity, which is not captured by traditional sentence-level augmentation methods such as paraphrasing or back-translation. |
Evaluating Embedding APIs for Information Retrieval (2023.acl-industry)
Copied to clipboard
Ehsan Kamalloo, Xinyu Zhang, Odunayo Ogundepo, Nandan Thakur, David Alfonso-hermelo, Mehdi Rezagholizadeh, Jimmy Lin
| Challenge: | a growing number of language models are limiting their access to the community . we evaluate existing APIs for domain generalization and multilingual retrieval . |
| Approach: | They evaluate semantic embedding APIs in retrieval scenarios to assess their capabilities . they use BEIR and MIRACL to re-rank BM25 results using the APIs . |
| Outcome: | The proposed model is based on semantic embedding APIs that build vector representations of a given text. |
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on web search rely on real-user experiments, which can be costly to scale up. |
| Approach: | They propose a user simulation framework with LLM-based agents that can generate unique user profiles at scale. |
| Outcome: | The proposed framework can generate unique user profiles at scale, leading to diverse search behaviors. |
ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) have demonstrated remarkable performance in document reranking tasks. |
| Approach: | They propose a two-stage training approach for document reranking using reinforcement learning and fine-grained score learning. |
| Outcome: | The proposed approach outperforms open-source and proprietary reranking models on BEIR benchmark. |
Alignment Annotation for Clinic Visit Dialogue to Clinical Note Sentence Language Generation (2020.lrec-1)
Copied to clipboard
| Challenge: | Despite advances in natural language processing, converting a clinic visit conversation into a clinical note is a largely unexplored area of research. |
| Approach: | They propose an annotation methodology that is content- and technique- agnostic while associating note sentences to sets of dialogue sentences. |
| Outcome: | The proposed method is content- and technique-agnostic while associating note sentences to sets of dialogue sentences. |
DelucionQA: Detecting Hallucinations in Domain-specific Question Answering (2023.findings-emnlp)
Copied to clipboard
Mobashir Sadat, Zhengyu Zhou, Lukas Lange, Jun Araki, Arsalan Gundroo, Bingqing Wang, Rakesh Menon, Md Parvez, Zhe Feng
| Challenge: | Hallucination is a well-known phenomenon in text generated by large language models . state-of-the-art LLMs still have a number of weaknesses, including the tendency to generate hallucinatory statements without considering the factuality . |
| Approach: | They propose a dataset that captures hallucinations made by retrieval-augmented LLMs . they propose to use these methods to help detect hallucinosity in QA tasks . |
| Outcome: | The proposed method captures hallucinations made by retrieval-augmented LLMs for QA tasks. |
RAMQA: A Unified Framework for Retrieval-Augmented Multi-Modal Question Answering (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing ranking methods rely on small encoder-based ranking models, which are incompatible with modern decoder--based generative large language models (LLMs) Existing methods based on small LLaVA rankers are incompatible with advanced LLMs. |
| Approach: | They propose a framework that combines learning-to-rank methods with generative permutation-enhanced ranking techniques. |
| Outcome: | The proposed framework improves on two benchmarks, WebQA and MultiModalQA, showing significant improvements over baselines. |
Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies suggest re-rankers by fine-tuning pre-trained language models . however, manual search for discrete prompts is expensive and sub-optimal in transferability . |
| Approach: | They propose a discrete prompt optimization method that guides the generated texts toward optimal prompts . they propose to use large-scale language models as a zero-shot re-ranker . |
| Outcome: | The proposed method improves the performance of the re-ranker against baselines and human prompts. |
ASK: Aspects and Retrieval based Hybrid Clarification in Task Oriented Dialogue Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Ambiguous user queries pose a challenge in task-oriented dialogue systems . Large Language Models (LLMs) rely on the top-k retrieved documents for clarification . traditional approaches lack principled mechanisms to determine when to use broad domain knowledge vs specific retrieved document context for clarification. |
| Approach: | They propose a hybrid approach that dynamically chooses between document-based or aspect-based clarification based on query ambiguity. |
| Outcome: | The proposed approach shows significant improvements over baselines on product troubleshooting and product search datasets. |
SCV: Light and Effective Multi-Vector Retrieval with Sequence Compressive Vectors (2025.coling-industry)
Copied to clipboard
| Challenge: | Recent advances in language models (LMs) have driven progress in information retrieval (IR), effectively extracting semantically relevant information. |
| Approach: | They propose a multi-vector retrieval framework with sequence compression vectors and a coarse-to-fine vector search framework that uses span compressive vectors to score documents. |
| Outcome: | The proposed approach achieves the fastest latency compared to other state-of-the-art models and can obtain competitive performance on both in-domain and out-of domain benchmark datasets. |
Personality Matters: User Traits Predict LLM Preferences in Multi-Turn Collaborative Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly integrated into everyday workflows . a recent study found that LLMs exhibit distinct personality-like traits that affect user engagement . |
| Approach: | They evaluated 32 LLM users for four collaborative tasks and found significant preferences . they found that rationalists preferred GPT-4, while idealists favored Claude 3.5 . |
| Outcome: | The results show that users with different personality traits prefer certain LLMs over others. |
PLOD: An Abbreviation Detection Dataset for Scientific Documents (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for abbreviation detection and extraction are limited. |
| Approach: | They propose to use a large-scale dataset for abbreviation detection and extraction that contains 160k+ segments automatically annotated with abbrevian and long forms. |
| Outcome: | The proposed dataset has an F1 score of 0.92 for abbreviations and 0.89 for detecting their corresponding long forms. |
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2023.findings-acl)
Copied to clipboard
Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | a new method for embedding text is developed for tasks that require specialized encoders . INSTRUCTOR is a single embedder that can generate text embeddables tailored to different tasks and domains based on instruction finetuning . |
| Approach: | They introduce a new method for computing text embeddings given task instructions . they first annotate instructions for 330 diverse tasks and train INSTRUCTOR on this multitask mixture . |
| Outcome: | The proposed method improves on 70 embedding evaluation tasks with fewer parameters than the previous best model. |
One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing sentence embedding methods lack the ability to capture the implicit semantics of sentences. |
| Approach: | They propose a sentence embedding method that assigns two embeddables to each sentence . one represents the explicit semantics and the other represents the implicit semantics . results show DualCSE can effectively encode both explicit and implicit meanings - they argue . |
| Outcome: | The proposed method can effectively encode both explicit and implicit meanings and improve the performance of the downstream task. |
Knowledge-Enhanced Named Entity Disambiguation for Short Text (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods for named entity disambiguation are weak for short text . performance of existing methods drops dramatically for short texts . |
| Approach: | They propose a knowledge-enhanced method for named entity disambiguation . they use factual knowledge graph and conceptual knowledge graph to provide additional knowledge . |
| Outcome: | The proposed method achieves significant improvement on a large manually annotated short-text dataset and the state-of-the-art on three standard datasets. |
Hengam: An Adversarially Trained Transformer for Persian Temporal Tagging (2022.aacl-main)
Copied to clipboard
| Challenge: | A wide array of natural language processing (NLP) applications relies on accurately identifying events and their respective occurrence times. |
| Approach: | They propose an adversarially trained transformer for Persian temporal tagging that can generalize over the HengamTagger’s rules. |
| Outcome: | The proposed tool outperforms state-of-the-art methods on a diverse and manually created dataset. |
EqualizeIR: Mitigating Linguistic Biases in Retrieval Models (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing information retrieval models show significant linguistic biases based on the linguistic complexity of queries. |
| Approach: | They propose a framework to mitigate linguistic biases in IR models by using a linguistically biased weak learner to capture biased queries and then train a robust model by regularizing and refining its predictions. |
| Outcome: | The proposed framework reduces performance disparities across simple and complex queries while improving overall retrieval performance. |
OMG-QA: Building Open-Domain Multi-Modal Generative Question Answering Systems (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to QA require multiple modalities and a broad pool of information sources to generate coherent answers. |
| Approach: | They propose a new resource to evaluate the effectiveness of question answering systems that perform retrieval augmented generation in scenarios that demand reasoning on multi-modal, multi-document contexts. |
| Outcome: | The proposed method evaluates question answering systems that perform retrieval augmented generation (RAG) in open-domain questions . it requires systems to navigate diverse modalities and a broad pool of information sources, making it uniquely challenging. |
HEAD-QA: A Healthcare Dataset for Complex Reasoning (P19-1)
Copied to clipboard
| Challenge: | Recent progress in question answering has been led by neural models, but current methods are too data intensive and weak. |
| Approach: | They propose a multi-choice question answering testbed to encourage research on complex reasoning. |
| Outcome: | The proposed dataset is useful as a benchmark for future work. |
Improving Absent Keyphrase Generation with Diversity Heads (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to generating keyphrases for a given text are limited to extracting only the keyphrase that is directly seen in the document. |
| Approach: | They propose to treat present keyphrase extraction as a sequence labeling problem and treat absent keyphrases together in a text-to-text generation framework during training. |
| Outcome: | The proposed model improves on the state-of-the-art for present keyphrase extraction and five datasets for absent keyphrase generation among the six English datasets. |
Exploring the Best Practices of Query Expansion with Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are foundational in language technologies, particularly in information retrieval (IR). |
| Approach: | They propose a framework that leverages large language models for query expansion . they use LLMs to generate multiple pseudo-references and integrate them with original queries . |
| Outcome: | The proposed framework enhances sparse and dense retrieval methods without pre-indexing. |
JTAV: Jointly Learning Social Media Content Representation by Fusing Textual, Acoustic, and Visual Features (C18-1)
Copied to clipboard
| Challenge: | Existing studies on learning social media content focus on single modal or bi-modal learning, but this approach is non-trivial and challenging because content is multi-modal and involves several types of data, including text, audio, and image. |
| Approach: | They propose to combine textual, acoustic, and visual information to learn social media content by fusing them jointly. |
| Outcome: | The proposed model outperforms the state-of-the-art approaches on real-world datasets by a large margin. |
Domain-matched Pre-training Tasks for Dense Retrieval (2022.findings-naacl)
Copied to clipboard
Barlas Oguz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Scott Yih, Sonal Gupta, Yashar Mehdad
| Challenge: | Existing approaches to improve performance of pre-training tasks are needed. |
| Approach: | They propose to pre-train large bi-encoder models on a recently released set of 65 millionsynthetically generated questions and 200 million post-comment pairs from a preexisting reddit conversation dataset. |
| Outcome: | The proposed model can be pre-trained on a set of 65 millionsynthetically generated questions and 200 million post-comment pairs from a preexisting dataset of Reddit conversations. |
Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-Lingual Semantic Discrimination (CLSD) is a lightweight evaluation task that requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors. |
| Approach: | They propose a lightweight task that requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors. |
| Outcome: | The proposed task requires only parallel sentences and a Large Language Model (LLM) to generate adversarial distractors. |
Leveraging a Bilingual Dictionary to Learn Wolastoqey Word Representations (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing word embeddings for lowresource languages require large corpora of running text to learn high quality representations. |
| Approach: | They leverage a bilingual dictionary to learn Wolastoqey word embeddings by encoding their corresponding English definitions into vector representations using pretrained English word and sequence representation models. |
| Outcome: | The proposed model outperforms baseline models without language-specific training or fine-tuning. |
Are Your Keywords Like My Queries? A Corpus-Wide Evaluation of Keyword Extractors with Real Searches (2025.coling-main)
Copied to clipboard
| Challenge: | Keyword Extraction (KE) is essential in Natural Language Processing (NLP) for identifying key terms that represent the main themes of a text. |
| Approach: | They propose to use real query data from Google Trends to evaluate keywords extracted from a text to capture users' top queries. |
| Outcome: | The proposed method can be used with both supervised and unsupervised KE approaches and shows that KeyBERT is the most effective in capturing users’ top queries. |
Beyond [CLS] through Ranking by Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on generative ranking models for Information Retrieval has focused on discriminative methods that learn a similarity function to compare questions and candidates answers. |
| Approach: | They propose to use a language model to train a ranking function that model the semantic similarity of documents and queries instead of discriminative ranking functions. |
| Outcome: | The proposed approaches are as effective as state-of-the-art discriminative models for the answer selection task and show unlikelihood losses are reduced for IR. |
When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets (2024.findings-eacl)
Copied to clipboard
| Challenge: | Using large language models (LMs) for query or document expansion can improve generalization in information retrieval. |
| Approach: | They conduct the first comprehensive analysis of large language models (LMs) for query or document expansion. |
| Outcome: | The proposed expansions improve retrieval performance for weaker models but harm stronger models. |
MatRank: Text Re-ranking by Latent Preference Matrix (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for text ranking have improved performance, but there are still challenges. |
| Approach: | They propose a method that learns to re-rank the text retrieved for a given query by learning to predict the most relevant passage based on a latent preference matrix. |
| Outcome: | The proposed method outperforms all prior methods on datasets with extensive results. |
XLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Temporal Expression Extraction (TEE) is essential for understanding time in natural language. |
| Approach: | They propose a framework for multilingual Temporal Expression Extraction that leverages pre-trained language models to prompt cross-language knowledge transfer from English to non-English languages. |
| Outcome: | The proposed framework outperforms the existing SOTA methods on French, Spanish, Portuguese, and Basque by large margins. |
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024.acl-long)
Copied to clipboard
Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zheng Liu, Ji-Rong Wen, Zhicheng Dou
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks, but their application to information retrieval tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. |
| Approach: | They propose to use instruction tuning to enhance LLMs' proficiency in IR tasks by combining a dataset with manually written templates to analyze the effects of instruction design, template diversity, few-shot demonstrations, and the volume of instructions. |
| Outcome: | The proposed model can be used to perform query understanding, document understanding, and query-document relationship understanding tasks. |
MedEureka: A Medical Domain Benchmark for Multi-Granularity and Multi-Data-Type Embedding-Based Retrieval (2025.findings-naacl)
Copied to clipboard
| Challenge: | Embedding-based retrieval (EBR) is a mainstream approach in information retrieval. |
| Approach: | They propose an enriched benchmark to evaluate retrieval capabilities of embedding models . they use four levels of granularity and six types of medical texts to prompt instruction-fine-tuned embeddable models. |
| Outcome: | The proposed benchmark evaluates the retrieval capabilities of embedding models with multi-granularity and multi-data types. |
A Survey on Recent Advances in Keyphrase Extraction from Pre-trained Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Keyphrase extraction is a key component in Natural Language Processing (NLP) systems for selecting a set of phrases from the document that could summarize the important information discussed in the source document. |
| Approach: | They propose to use supervised and unsupervised keyphrase extraction techniques to investigate the state-of-the-art models for keyphrase extracting. |
| Outcome: | The proposed keyphrase extraction system can significantly accelerate the speed of retrieval and help people get first-hand information from a long document quickly and accurately. |
Long Context Modeling with Ranked Memory-Augmented Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) face a fundamental limitation in processing long-context scenarios due to quadratic complexity of attention mechanisms and increasing memory demands during generation. |
| Approach: | They propose a framework that dynamically ranks memory entries based on relevance . ERMAR employs a relevance scoring mechanism and a pointwise re-ranking model for key-value embeddings . |
| Outcome: | The proposed framework achieves state-of-the-art performance on benchmarks . it uses historical usage patterns and adaptive retrieval to improve performance . |
Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models (2025.naacl-long)
Copied to clipboard
Hengyi Wang, Haizhou Shi, Shiwei Tan, Weiyi Qin, Wenyuan Wang, Tunyu Zhang, Akshay Nambi, Tanuja Ganu, Hao Wang
| Challenge: | Multimodal Large Language Models have shown significant promise in various applications, but a comprehensive evaluation of their long-context capabilities remains underexplored. |
| Approach: | They propose a benchmark to assess the long-context capabilities of multimodal large language models. |
| Outcome: | The proposed benchmark compared MLLMs with API-based and open-source models in a long-context scenario. |
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models (C18-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. |
| Approach: | They propose to use recurrent neural networks to generate NERs over characters, sub-words and/or word embeddings to improve named entity recognition. |
| Outcome: | The proposed architectures are better than those based on feature engineering and other supervised or semi-supervised learning algorithms. |
Efficiency-Effectiveness Reranking FLOPs for LLM-based Rerankers (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing studies evaluate the efficiency of LLM-based rerankers using proxy metrics such as latency and the number of forward passes. |
| Approach: | They propose to use a large language model to evaluate the efficiency of LLM-based rerankers . they propose to measure ranking quality and query processing efficiency using an interpretable FLOPs estimator . |
| Outcome: | The proposed metrics evaluate LLM-based rerankers with different architectures without running any experiments. |
BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filtering (2024.lrec-main)
Copied to clipboard
| Challenge: | Bangla is underrepresented in KGs due to lack of comprehensive datasets, encoders, NER models, part-of-speech taggers, and lemmatizers. |
| Approach: | Bangla is underrepresented in KGs due to lack of comprehensive datasets, encoders, NER models, part-of-speech taggers, and lemmatizers. authors propose a framework that can automatically construct Bengali KG from any Bangla text. |
| Outcome: | The proposed framework can automatically construct Bengali KGs from any Bangla text. |
Service-oriented Text-to-SQL Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | TEXT2SQL models focus on generating complex SQL query in a precise and complete fashion . current models focus only on executing complex queries in production environment . TExT2sql models are limited by human labor and limited by predefined template . |
| Approach: | They propose a text-to-sql parser that translates natural language utterance to SQL query . the framework enables flexibly access database than rigid API in the application . |
| Outcome: | The proposed model outperforms baseline models in the WikiSQL task by 13% error reduction. |
Database Search vs. Information Retrieval: A Novel Method for Studying Natural Language Querying of Semi-Structured Data (2020.lrec-1)
Copied to clipboard
| Challenge: | Traditionally, technicians had crafted complicated search commands in languages such as SQL to retrieve results from relational databases. |
| Approach: | They propose to map a database schema to a keyword-enabled relational database system that can be directly compared with information retrieval. |
| Outcome: | The proposed approach compares SODA, a keyword-enabled relational database system, against the Terrier information retrieval system. |
DeepMaven: Deep Question Answering on Long-Distance Movie/TV Show Videos with Multimedia Knowledge Extraction and Synthesis (2023.eacl-main)
Copied to clipboard
| Challenge: | Long video content understanding poses a challenging set of research questions as it involves long-distance, cross-media reasoning and knowledge awareness. |
| Approach: | They propose a framework which extracts events, entities, and relations from the rich multimedia content in long videos to pre-construct movie knowledge graphs. |
| Outcome: | The proposed framework performs competitively for both the new DeepMovieQA and the pre-existing MovieQA dataset. |
Intent Contrastive Learning Based on Multi-view Augmentation for Sequential Recommendation (2025.coling-main)
Copied to clipboard
Bo Pei, Yingzheng Zhu, Guangjin Wang, Huajuan Duan, Wenya Wu, Fuyong Xu, Yizhao Zhu, Peiyu Liu, Ran Lu
| Challenge: | Existing work on intent-related models fails to capture long-term dependencies in user behavior and fails to effectively utilize item relevance. |
| Approach: | They propose a sequential recommendation framework that combine temporal variability with position encoding that has extrapolation properties to encode sequences, thereby expanding the model’s view of user behavior. |
| Outcome: | The proposed model improves on three real datasets by 0.8% to 14.7% compared to baselines. |
Corpus Poisoning via Approximate Greedy Gradient Descent (2025.findings-acl)
Copied to clipboard
| Challenge: | Dense retrievers have been shown to be vulnerable to corpus poisoning attacks . a malicious user injects a small fraction of adversarial passages into the retrieval corpus to trick the system into returning these passages among the top-ranked results for a broad set of user queries. |
| Approach: | They propose an attack on dense retrieval systems based on the widely used HotFlip method for efficiently generating adversarial passages. |
| Outcome: | The proposed method achieves a high success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains. |
Dealing with Typos for BERT-based Passage Retrieval and Ranking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to passage retrieval and ranking rely on pre-trained deep language models that model the semantic matching between queries and passages. |
| Approach: | They propose a typos-aware training framework for DR and BERT to address this issue. |
| Outcome: | The proposed models respond and adapt to keyword typos occurring in queries, and significantly improve their retrieval and ranking effectiveness. |
UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrievers (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing information retrieval models assume a homogeneous structure for knowledge sources and user queries, limiting their applicability in real-world settings. |
| Approach: | They propose a unified instruction-aware heterogeneous knowledge retriever that builds a heterogenous retrieval space for heterogenized knowledge and follows diverse user instructions to retrieve knowledge in specified types. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on CompMix-IR . it achieves 6.36% relative improvements and 54.23% relative improvements . |
WIKIR: A Python Toolkit for Building a Large-scale Wikipedia-based English Information Retrieval Dataset (2020.lrec-1)
Copied to clipboard
| Challenge: | ad-hoc information retrieval methods usually require large amounts of annotated data to be effective. |
| Approach: | They propose an open-source toolkit to automatically build large-scale English information retrieval datasets based on Wikipedia. |
| Outcome: | The proposed toolkit builds large-scale English information retrieval datasets based on Wikipedia with 59,252 queries and 2,617,003 pairs. |
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (D19-1)
Copied to clipboard
| Challenge: | Recent advances in representation learning have separated progress in both IR and MC . few studies have examined the relationship between retrieval and comprehension at different levels of granularity for development of MRS systems. |
| Approach: | They propose a simple yet effective pipeline system with consideration on hierarchical semantic retrieval at both paragraph and sentence level and their potential effects on the downstream task. |
| Outcome: | The proposed system achieves state-of-the-art on the leaderboard test sets of both FEVER and HOTPOTQA. |
Large Scale Multi-Lingual Multi-Modal Summarization Dataset (2023.eacl-main)
Copied to clipboard
| Challenge: | a large dataset of document-image pairs and annotated multi-modal summarization data is needed for multi-lingual modeling . encoder-decoder models represent information comprising multiple modalities. |
| Approach: | They propose to use a multi-lingual summarization dataset to analyze multi-modal summarizing using multi-linguistic annotated data. |
| Outcome: | The proposed dataset is the largest multi-lingual multi-modal summarization dataset for 13 languages and consists of cross-lingual summarizing data for 2 languages. |
Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent large language models demonstrate remarkable abilities in responding to queries in diverse languages, but their ability to handle long multilingual contexts is unexplored. |
| Approach: | They propose a multilingual Needle-in-a-Haystack (MLNeedle) test to assess a model's ability to retrieve relevant information from a collection of multilingual distractor texts. |
| Outcome: | The proposed model performance is the lowest when the needle is in a language outside the English language family and (ii) located in the middle of the input context. |
An Empirical Study of Position Bias in Modern Information Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new evaluation framework is used to assess the extent and impact of position bias in information retrieval. |
| Approach: | They introduce a position-aware retrieval benchmark and a diagnostic metric to quantify position bias . they compare models with BM25, dense embedding models, ColBERT-style late-interaction models . |
| Outcome: | The proposed framework evaluates retrieval models for position bias from a worst-case perspective. |
Disentangling Questions from Query Generation for Task-Adaptive Retrieval (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work generates synthetic queries from domain-specific documents to jointly train the retriever. |
| Approach: | They propose a query generator that better adapts to wide search intents expressed in the BeIR benchmark. |
| Outcome: | The proposed query generator outperforms baselines and existing models on tasks with underexplored intents while using a query generator 47 times smaller than the previous state-of-the-art. |
Semantic Parsing for Technical Support Questions (C18-1)
Copied to clipboard
| Challenge: | Automating technical support is a task of providing answers to complex problems . traditional approaches to this task rely on information retrieval and are keyword based . |
| Approach: | They propose a semantic parsing approach that uses grammatical structure to extract technical questions' attributes as a baseline and a CRF-based model that can improve performance in the presence of annotated data. |
| Outcome: | The proposed model outperforms retrieval baselines in annotated data for training. |
SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | SUQL is a conversational language that supports the generality of hybrid data access for large knowledge corpora. |
| Approach: | They propose a conversational agent that supports the full generality of hybrid data access for large knowledge corpora using SUQL. |
| Outcome: | The proposed language can handle hybrid data sources. |
Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing text semantic matching models do not provide granularity for text comparison. |
| Approach: | They propose a simple yet effective training strategy for text semantic matching by disentangling keywords from intents. |
| Outcome: | The proposed approach achieves stable performance improvements against a wide range of models on three benchmarks. |
QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for large-scale query-document retrieval are expensive and require sparse handcrafted features. |
| Approach: | They propose a quadrupletBERT model for effective and efficient retrieval using pre-trained language models like BERT. |
| Outcome: | The proposed model improves retrieval phase and leverages distances between simple negative and hard negative instances to obtain better embeddings. |
Test-time Corpus Feedback: From Retrieval to RAG (2026.findings-eacl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) pipelines treat retrieval and reasoning as isolated components, limiting performance on complex tasks. |
| Approach: | They propose to integrate large language models with retrieval to improve query quality . they also propose to use feedback to improve the query, retrieved context, or document pool . |
| Outcome: | The proposed methods bridge IR and NLP perspectives and highlight retrieval as a dynamic, learnable component of end-to-end RAG systems. |
Extending Context Window of Large Language Models via Semantic Compression (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models rely on a quadratic computation to generate long texts . current models impose limitations on the length of text inputs . |
| Approach: | They propose a semantic compression method that extends the context window of large language models . the method reduces the semantic redundancy of long inputs before passing them to the LLMs . |
| Outcome: | The proposed method extends the context window of large language models across tasks . it exhibits consistent fluency in text generation while reducing associated computational overhead. |
Exploring Diverse Expressions for Paraphrase Generation (D19-1)
Copied to clipboard
| Challenge: | Existing neural paraphrase generation methods focus on single paraphrases while ignoring the fact that diversity is essential for enhancing generalization capability and robustness of downstream applications. |
| Approach: | They propose a novel approach with two discriminators and multiple generators to generate a variety of different paraphrases. |
| Outcome: | The proposed model gains significant diversity and improves quality over state-of-the-art datasets. |
Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankers (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) are more efficient in natural language understanding tasks. |
| Approach: | They evaluate large language models (LLMs) using a TREC Fair Ranking dataset . they assess fairness from both user and content perspectives . |
| Outcome: | The proposed model outperforms the existing models in the fair ranking task. |
tRAG: Term-level Retrieval-Augmented Generation for Domain-Adaptive Retrieval (2025.naacl-long)
Copied to clipboard
| Challenge: | Neural retrieval models suffer when there is a domain shift between training and test data distributions. |
| Approach: | They propose to generate domain-adapted pseudo-queries using large language models (LLMs) to improve term recall of unseen query terms by using term-level Retrieval-Augmented Generation (tRAG). |
| Outcome: | The proposed method significantly improves recall for unseen terms by 10.6% and outperforms LLM and retrieval-augmented generation baselines on overall retrieval performance. |
Time-aware ReAct Agent for Temporal Knowledge Graph Question Answering (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing solutions for temporal knowledge graph question answering lack sufficient temporal constraints in retrieval process. |
| Approach: | They propose a temporal knowledge graph question answering framework that integrates temporal constraints into information retrieval. |
| Outcome: | The proposed framework achieves a 41.3% improvement over the baseline model and a 32.2% gain compared to the Abstract Reasoning Induction (ARI) method. |
Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent supervised neural models have greatly promoted the development of topic segmentation, but the deeper relationship between coherence and topic segmenting is underexplored. |
| Approach: | They propose to use topic-aware Sentence Structure Prediction and Contrastive Semantic Similarity Learning to capture coherence from logical structure and semantic similarity perspectives to further improve topic segmentation performance. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on WIKI-727K and achieves an average relative reduction of 4.3% on Pk on WikiSection. |
Modularized Transfomer-based Ranking Framework (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent innovations in Transformer-based ranking models have advanced the state-of-the-art in information retrieval. |
| Approach: | They propose to modularize a Transformer ranker into separate modules for text representation and interaction. |
| Outcome: | The proposed model is faster than previous models and is easier to interpret and understand. |
C-STS: Conditional Semantic Textual Similarity (2023.emnlp-main)
Copied to clipboard
Ameet Deshpande, Carlos Jimenez, Howard Chen, Vishvak Murahari, Victoria Graf, Tanmay Rajpurohit, Ashwin Kalyan, Danqi Chen, Karthik Narasimhan
| Challenge: | Semantic textual similarity (STS) is a cornerstone task in natural language processing, but it is inherently ambiguous. |
| Approach: | They propose a task called conditional STS which measures similarity conditioned on an aspect elucidated in natural language. |
| Outcome: | The proposed task reduces subjectivity and ambiguity and enables fine-grained similarity evaluation using diverse conditions. |
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval (2023.acl-long)
Copied to clipboard
| Challenge: | Dense retrieval has shown promise in the first-stage retrieval process when trained on in-domain labeled datasets. |
| Approach: | They propose a method to capture matching signal to improve generalization of dense retrieval by capturing matching signal between two texts. |
| Outcome: | The proposed method can be combined with different training methods to improve generalization ability without additional inference overhead and target domain data. |
What Do Indonesians Really Need from Language Technology? A Nationwide Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite efforts to develop NLP for Indonesia’s 700+ local languages, progress remains costly due to the need for direct engagement with native speakers. |
| Approach: | They conduct a nationwide survey to assess the actual needs of native Indonesian speakers. |
| Outcome: | The findings indicate that addressing language barriers is the most critical priority . concerns around privacy, bias, and the use of public data highlight the need for greater transparency and clear communication to support broader AI adoption. |
From Token to Action: State Machine Reasoning to Mitigate Overthinking in Information Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting often leads to overthinking in large language models . redundant trajectories that revisit similar states and misguided reasoning that diverges from user intent are two key challenges in information retrieval. |
| Approach: | They propose a transition-based reasoning framework that supports early stopping and fine-grained control. |
| Outcome: | The proposed framework improves retrieval performance by 3.4% while reducing token usage by 74.4%. |
A Text is Worth Several Tokens: Text Embedding from LLMs Secretly Aligns Well with The Key Tokens (2025.acl-long)
Copied to clipboard
| Challenge: | et al., 2023) show that text embeddings from large language models can be aligned with key tokens in input text. |
| Approach: | They propose a sparse retrieval method based on aligned tokens for large language models . they show that this phenomenon is universal and is not affected by model architecture . |
| Outcome: | The proposed method can achieve 80% of the dense retrieval effect of the same model while reducing the computation significantly. |
Text Segmentation by Cross Segment Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Document and discourse segmentation are two fundamental NLP tasks pertaining to breaking up text into constituents. |
| Approach: | They propose three transformer-based NLP models that break up text into constituents and compare them to previous approaches. |
| Outcome: | The proposed architectures reduce errors by a large margin on three datasets and improve performance on real-world datasets. |
Programming in Natural Language with fuSE: Synthesizing Methods from Spoken Utterances Using Deep Natural Language Understanding (2020.acl-main)
Copied to clipboard
| Challenge: | a new approach to teach new functions from natural language is needed to make intelligent systems programmable in everyday language. |
| Approach: | They propose to use natural language to teach intelligent systems new functions . fuSE synthesizes method signatures and API calls from spoken utterances . |
| Outcome: | The proposed system synthesizes 84.6% of method signatures and 79.2% of API calls correctly on unseen dataset. |
HEISIR: Hierarchical Expansion of Inverted Semantic Indexing for Training-free Retrieval of Conversational Data using LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for retrieving dialogue data often face challenges in capturing semantic intent or require extensive labeling and fine-tuning. |
| Approach: | They propose a framework that enhances semantic understanding in conversational data retrieval by optimizing data ingestion. |
| Outcome: | The proposed framework outperforms fine-tuned models in retrieval and intent analysis. |
Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used as rerankers, but their ranking behavior can be steered by small, natural-sounding prompts. |
| Approach: | They propose a token optimization method that crafts concise textual perturbations to consistently promote a target item in LLM-generated rankings. |
| Outcome: | The proposed method outperforms state-of-the-art base-lines and is hard to detect. |
HOTTER: Hierarchical Optimal Topic Transport with Explanatory Context Representations (2021.findings-emnlp)
Copied to clipboard
Sabine Wehnert, Christian Scheel, Simona Szakács-Behling, Maret Nieländer, Patrick Mielke, Ernesto William De Luca
| Challenge: | Natural language processing (NLP) is often the backbone of today’s systems for user interactions, information retrieval and others. |
| Approach: | They propose an extension to a specific emerging hybrid document distance metric which combines topic models and word embeddings. |
| Outcome: | The proposed method is competitive on public datasets and the language model BERT is used for a document categorization task. |
CLERC: A Dataset for U. S. Legal Case Retrieval and Retrieval-Augmented Analysis Generation (2025.findings-naacl)
Copied to clipboard
Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme
| Challenge: | a dataset of case law is used to train and evaluate models for writing legal analyses . current approaches struggle to find relevant cases and generate legal analyses, authors say . |
| Approach: | They build a dataset of case law to support information retrieval and retrieval-augmented generation. |
| Outcome: | The proposed dataset supports two important backbone tasks: retrieval (IR) and retrieval-augmented generation (RAG). |
Insights into LLM Long-Context Failures: When Transformers Know but Don’t Tell (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit positional bias, struggling to utilize information from the middle or end of long contexts. |
| Approach: | They propose to examine LLMs' long-context generalizations by probing their hidden representations. |
| Outcome: | The proposed models excel at processing extended contexts while preserving their positional bias. |
IterCQR: Iterative Conversational Query Reformulation with Retrieval Guidance (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for conversational query reformulation depend on human annotations. |
| Approach: | They propose a method that reformulates context-dependent conversational queries without relying on human rewrites. |
| Outcome: | The proposed method shows state-of-the-art performance on two widely-used datasets. |
Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Enterprise systems are crucial for enhancing productivity and strategic growth, but data is fragmented across multiple sources and access controls are complex. |
| Approach: | They propose a benchmark that simulates enterprise settings with 500 diverse tasks . they show that even the most capable models achieve only 41.8% task completion . |
| Outcome: | The proposed benchmark shows that even the most capable models achieve only 41.8% task completion. |
Embedding Meta-Textual Information for Improved Learning to Rank (2020.coling-main)
Copied to clipboard
| Challenge: | a neural representation learning approach has not been extended to meta-textual information that is readily available for many IR tasks. |
| Approach: | They propose a framework that learns embeddings for meta-textual categories and optimizes a pairwise ranking objective for improved matching based on combined embedds of textual and meta-tactile information. |
| Outcome: | The proposed framework improves cross-lingual retrieval in the Wikipedia domain and Patent domain. |
FIRST: Faster Improved Listwise Reranking with Single Token Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing listwise LLMs lack efficiency as they provide ranking output in the form of a generated ordered sequence of candidate passage identifiers. |
| Approach: | They propose a listwise LLM reranking approach that leverages the first generated identifier to obtain a ranked ordering of the candidates. |
| Outcome: | The proposed approach accelerates inference by 50% while maintaining robust ranking performance with gains across BEIR benchmark. |
WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)
Copied to clipboard
Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou
| Challenge: | Long-form question answering requires two procedures: information retrieval and information synthesis. |
| Approach: | They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time . |
| Outcome: | The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset . |
Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have improved IE, but their potential for ATE has not been explored. |
| Approach: | They propose a retrieval-based prompting strategy that selects demonstrations according to syntactic rather than semantic similarity in a few-shot setting. |
| Outcome: | The proposed method improves performance on three specialized ATE benchmarks. |
Enhanced Facet Generation with LLM Editing (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have shown that search engines can recognize facets of a user's query. |
| Approach: | They propose to use large language models to enhance the facets of a query to generate facets from a search engine. |
| Outcome: | The proposed model can predict facets by taking only queries as input without a search engine. |
100,000 Podcasts: A Spoken English Document Corpus (2020.coling-main)
Copied to clipboard
Ann Clifton, Sravana Reddy, Yongze Yu, Aasish Pappu, Rezvaneh Rezapour, Hamed Bonab, Maria Eskevich, Gareth Jones, Jussi Karlgren, Ben Carterette, Rosie Jones
| Challenge: | Podcasts are a large and growing repository of spoken audio. |
| Approach: | They propose to use podcasts as a resource for speech processing and linguistics . they use a corpus of 100,000 podcasts to study the complexity of the domain . |
| Outcome: | The Spotify Podcast Dataset is the largest corpus of transcribed speech data . the dataset contains 60,000 hours of podcasts, with a range of genres and styles . |
Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling (D19-1)
Copied to clipboard
| Challenge: | Existing techniques for relevance and semantic matching cannot be easily adapted to the other. |
| Approach: | They propose a model that incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness. |
| Outcome: | The proposed model incorporates a hybrid encoder module, a relevance matching module and co-attention mechanisms that capture context-aware semantic relatedness. |
A Joint Approach to Compound Splitting and Idiomatic Compound Detection (2020.lrec-1)
Copied to clipboard
| Challenge: | Compounding is a common word-formation process in Germanic languages . high productivity and low corpus frequency of compounds increase vocabulary size . |
| Approach: | They develop a deep learning-based approach to noun compound splitting and idiomatic compound detection for the German language. |
| Outcome: | The proposed approach outperforms the current state of the art in noun compound splitting and idiomatic compound detection for the German language. |
MQuinE: a Cure for “Z-paradox” in Knowledge Graph Embedding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from Z-paradox, a deficiency in expressiveness . Embedding-based models map each entity and relation into a vector or matrix . |
| Approach: | They propose a new knowledge graph embedding model that does not suffer from Z-paradox while preserves strong expressiveness to model various relation patterns with theoretical justification. |
| Outcome: | The proposed model outperforms existing models on link prediction tasks while maintaining strong expressiveness. |
Pareto Optimal Learning for Estimating Large Language Model Errors (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for improving the correctness of LLMs include prompt engineering, retrieval methods, and a generative model. |
| Approach: | They propose a method that generates a risk score to estimate the probability of error in an LLM by integrating multiple sources of information. |
| Outcome: | The proposed method is well correlated with the true LLM error rate, thus facilitating error correction. |
Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Embeddings from Large Language Models (LLMs) have emerged as critical components in information retrieval applications. |
| Approach: | They propose a tuning framework for the customization of LLM embeddings. |
| Outcome: | The proposed framework reduces embedding dimensions while maintaining comparable performance levels. |
Event-enhanced Retrieval in Real-time Search (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing embedding-based retrieval models face the "semantic drift" problem . a low adoption rate of retrieval results is evident in real-time search scenarios . |
| Approach: | They propose an embedding-based retrieval approach that enhances real-time retrieval performance by adding contrastive learning to the dual-encoder model. |
| Outcome: | The proposed approach improves the dual-encoder model of traditional EBR. |
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that large language models (LLMs) rank documents based on the probability of generating the query given the content of a document. |
| Approach: | They propose a ranking system that integrates LLMs with a hybrid zero-shot retriever. |
| Outcome: | The proposed system shows exceptional ranking in both zero-shot and few-shot scenarios. |
The Ontology of Bulgarian Dialects – Architecture and Information Retrieval (2020.lrec-1)
Copied to clipboard
| Challenge: | The ontology of the Bulgarian Dialects demonstrates a novel usage of ontological modelling for the purposes of dialect digital archiving and information processing. |
| Approach: | They propose to use ontological modelling to capture the geographical distribution and diagnostic features of the Bulgarian dialects for digital archiving and information processing. |
| Outcome: | The ontology incorporates information on the dialects of the Bulgarian language and includes data from 84 dialects spoken not only on the territory of the Republic of Bulgaria, but also abroad. |
MICRON: Multigranular Interaction for Contextualizing RepresentatiON in Non-factoid Question Answering (D19-1)
Copied to clipboard
| Challenge: | Existing approaches for non-factoid question answering can be categorized into representation and interaction focused approaches. |
| Approach: | They propose a novel approach which derives contextualized uni-gram representation from n-grams. |
| Outcome: | The proposed approach achieves state-of-the-art in two public non-factoid question answering datasets. |
Annotating Attribution Relations in Arabic (L18-1)
Copied to clipboard
| Challenge: | Current studies focus on using lexical terms in long texts to verify author identity. |
| Approach: | They propose to annotate attributed arguments to the source in Arabic news with required syntactical and semantic features with required features. |
| Outcome: | The proposed method is applied to Arabic news and is compared with existing tools and methods. |
AlignScore: Evaluating Factual Consistency with A Unified Alignment Function (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate factual consistency of text depend on limited data . e.g., generated text can contain factual inconsistencies that are irrelevant to context . |
| Approach: | They propose a new holistic metric that measures factual inconsistencies . they use 4.7M training examples from 7 well-established tasks . |
| Outcome: | The proposed metric outperforms existing metrics on 22 datasets and matches or outperFORMs them. |
A Modular Architecture for Unsupervised Sarcasm Generation (D19-1)
Copied to clipboard
| Challenge: | Existing systems for sarcasm generation are elusive due to the fact that both selection of contents and training of sarcasm are based on the same data. |
| Approach: | They propose a framework that takes a literal negative opinion as input and translates it into a sarcastic version. |
| Outcome: | The proposed system outperforms baselines built using known unsupervised statistical and neural machine translation and style transfer techniques. |
PQR: Improving Dense Retrieval via Potential Query Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing training data is sparse, with each document associated with one or a few labeled queries. |
| Approach: | They propose a training-free potential query retrieval framework to address this problem . they use a Gaussian mixture distribution to model all potential queries for a document . |
| Outcome: | The proposed method is able to capture comprehensive semantic information from a document with multiple queries. |
Search-Adaptor: Embedding Customization for Information Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to embed text in large language models are limited to zero-shot setups and can be integrated with any LLM. |
| Approach: | They propose a method for customizing LLMs for information retrieval by modifying the embeddings generated by pre-trained LLM models and can be integrated with any LLM. |
| Outcome: | The proposed method improves performance on English, multilingual, and multimodal retrieval datasets by 5% over 14 BEIR datasets. |
A Two-Stage Adaptation of Large Language Models for Text Ranking (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models (PLMs) have significantly improved ranking performance in text ranking tasks. |
| Approach: | They propose a two-stage progressive paradigm to better adapt LLMs to text ranking by conducting continual pre-training on a large weakly-supervised corpus and performing SFT on high-quality data. |
| Outcome: | The proposed approach outperforms previous methods on in- and out-domain scenarios. |
Analyze, Generate and Refine: Query Expansion with LLMs for Zero-Shot Open-Domain QA (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods like GAR and EAR rely heavily on supervised training and struggle to maintain effectiveness across domains and datasets. |
| Approach: | They propose a QE approach based on a three-step prompting strategy to enhance query expansion by broadening the scope of queries with additional relevant texts. |
| Outcome: | The proposed approach outperforms state-of-the-art methods in out-domain zero-shot scenarios and outperformed existing methods in end-to-end evaluations. |
LAiW: A Chinese Legal Large Language Models Benchmark (2025.coling-main)
Copied to clipboard
Yongfu Dai, Duanyu Feng, Jimin Huang, Haochen Jia, Qianqian Xie, Yifang Zhang, Weiguang Han, Wei Tian, Hao Wang
| Challenge: | Xie et al., 2023) show that large language models (LLMs) can generate legal text, but lack the legal syllogism . legal experts are cautious about their practical application due to the opaque nature of the LLMs. |
| Approach: | They propose a Chinese legal LLM benchmark structured around the legal syllogism . they evaluate LLMs across three levels of capability, each reflecting a more complex stage of legal . |
| Outcome: | The proposed benchmark identifies that LLMs lack the legal syllogism, which hinders trust and understanding from legal experts. |
MESAQA: A Dataset for Multi-Span Contextual and Evidence-Grounded Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing question answering systems focus on extracting answers from single spans, but real-world scenarios require synthesizing information from multiple spans. |
| Approach: | They propose a dataset that leverages the MASH-QA dataset and large language models (LLMs) to ensure that each Q/A pair requires considering all selected spans. |
| Outcome: | The proposed method enables the model to answer multiple Q/A pairs in a single span, while ensuring that all selected spans are considered. |
CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models excel in simple tasks such as generating standalone code units, but real-world software development often involves complex code repositories with complex dependencies and extensive documentation. |
| Approach: | They propose a novel LLM-based agent framework that employs external tools for effective repo-level code generation. |
| Outcome: | The proposed framework outperforms commercial products like Github Copilot in the humanEval benchmark and shows that it is adaptable and efficient across multiple code generation tasks. |
A Generative Framework for Personalized Sticker Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing relevance-based generative retrieval methods lack personalization, leading to a mismatch between diverse user expectations and the retrieved results. |
| Approach: | They propose a representation learning model that learns discriminative user representations to encode user-specific sticker preferences. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in generating relevant stickers for queries. |
Exploring the Practicality of Generative Retrieval on Dynamic Corpora (2024.emnlp-main)
Copied to clipboard
| Challenge: | a lack of comprehensive comparison between GR and Dual Encoders in IR systems is limiting . GR is more adaptable to evolving knowledge (4–11%), robust in learning knowledge with temporal information, and efficient in terms of inference FLOPs (x2), indexing time (x6) and storage footprint (x4) |
| Approach: | They propose to use autoregressive language models to perform information retrieval (IR) their results highlight the potential of GR for future use in practical IR systems . |
| Outcome: | The proposed model is more adaptable to evolving knowledge (4–11%), robust in learning knowledge with temporal information, efficient inference FLOPs (x2), indexing time (x6), and storage footprint (x4) compared to the most common model, Dual Encoder (DE). |
Efficient Document Embeddings via Self-Contrastive Bregman Divergence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Despite recent advances in transformer-based sentence encoders, the encoding of long documents (Ks of words) is still challenging with respect to both efficiency and quality considerations. |
| Approach: | They propose to combine a self-contrastive siamese network and a convex neural Bregman divergence network to train longfomer-based document encoders using an unsupervised contrastive learning method. |
| Outcome: | The proposed model outperforms baseline models on three long document topic classification tasks from the legal and biomedical domains. |
History-Aware Conversational Dense Retrieval (2024.findings-acl)
Copied to clipboard
| Challenge: | Current approaches for conversational dense retrieval rely on fine-tuning a pre-trained ad-hoc retriever, which can be lengthy and noisy. |
| Approach: | They propose a context-denoised query reformulation and automatic mining of supervision signals based on historical turns. |
| Outcome: | The proposed system improves on two public conversational search datasets. |
IR2: Information Regularization for Information Retrieval (2024.lrec-main)
Copied to clipboard
| Challenge: | Effective information retrieval (IR) in settings with limited training data remains a challenging task. |
| Approach: | They propose a technique for reducing overfitting during synthetic data generation . they use DORIS-MAE, ArguAna, and WhatsThatBook as examples . |
| Outcome: | The proposed technique outperforms previous methods and reduces cost by 50% on three recent IR tasks characterized by complex queries. |
Unsupervised Named Entity Disambiguation for Low Resource Domains (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to Named Entity Disambiguation (NED) are inefficient for domain specific tasks such as searching, question answering and information extraction. |
| Approach: | They propose a unsupervised approach leveraging the concept of Group Steiner Trees which can identify the most relevant candidate for entity disambiguation using contextual similarities across candidate entities for all the mentions present in a document. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods by more than 40% in terms of Precision@1 and Hit@5 across various domain-specific datasets. |
Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memory (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for embodied reasoning are coarse-grained and expensive . branch-and-browse framework enables fine-grounded, memory-guided, and efficient multi-branch reasoning. |
| Approach: | They propose a framework that unifies structured reasoning-acting, contextual memory, and efficient execution. |
| Outcome: | The proposed framework achieves task success rate of 35.8% and reduces execution time by up to 40.4% relative to state-of-the-art methods. |
Predicting Item Survival for Multiple Choice Questions in a High-Stakes Medical Exam (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods of pretesting are costly and require a large pool of test questions to be replenished, updated and expanded over time. |
| Approach: | They propose to automatically predict an item's probability to "survive" pretesting by embedding new items within a live exam and analyzing the responses. |
| Outcome: | The proposed method is based on human-produced MCQs for a medical exam and shows that survival is modelled through linguistic features and embedding types and features inspired by information retrieval. |
A French Corpus for Semantic Similarity (2020.lrec-1)
Copied to clipboard
| Challenge: | Semantic textual similarity is a subtask of Natural Language Processing. |
| Approach: | They propose to use an annotation corpus for French to assess semantic similarity . they use an annotated corpus with 1,010 sentence pairs with five annotators . |
| Outcome: | The proposed corpus for French is the first that we know of. |
TRIAL: Token Relations and Importance Aware Late-interaction for Accurate Text Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Late-interaction based multi-vector retrieval systems rely on a naive summation of token-level similarity scores . this leads to inaccurate relevance estimation due to tokenization of semantic units and the influence of low-content words. |
| Approach: | They propose a late-interaction-based multi-vector retrieval system that uses token relations and token importance in relevance scoring. |
| Outcome: | Extensive tests show that TRIAL achieves state-of-the-art accuracy compared to existing methods. |
RISE: Leveraging Retrieval Techniques for Summarization Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Summarization evaluation approaches have relied on ROUGE for summarization, but they fall short of human evaluations. |
| Approach: | They propose a new approach to evaluate summaries by leveraging retrieval techniques . they use a dual-encoder retrieval setup to train a retrieval task . |
| Outcome: | The proposed method outperforms existing methods on two document summarization benchmarks and a long document summmarization test. |
Filling the Temporal Void: Recovering Missing Publication Years in the Project Gutenberg Corpus Using LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Currently, there is no publicly available corpus for diachronic text analysis due to the lack of accurate temporal metadata. |
| Approach: | They propose to add missing temporal metadata to the Gutenberg corpus by using open web, Wikipedia, and Open Library API sources. |
| Outcome: | The proposed corpus includes 53,774 books with a total of 3.8 billion tokens in 11 languages, produced between 1600 and 2000. |
RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models (2025.acl-long)
Copied to clipboard
Hieu Tran, Zonghai Yao, Zhichao Yang, Junda Wang, Yifan Zhang, Shuo Han, null Feiyun Ouyang, Hong Yu
| Challenge: | Existing work aims to improve reasoning accuracy and factual integrity across large language models for knowledge-intensive tasks such as medical and commonsense reasoning. |
| Approach: | They propose a versatile extension to the mutual reasoning framework (rStar) that enhances reasoning accuracy and factual integrity across large language models. |
| Outcome: | The proposed extension to the mutual reasoning framework improves reasoning accuracy and factual integrity across large language models for complex, knowledge-intensive tasks. |
An Empirical Comparison of LM-based Question and Answer Generation Methods (2023.findings-acl)
Copied to clipboard
| Challenge: | Question and answer generation (QAG) is a task of generating question-answer pairs given a context. |
| Approach: | They propose to leverage sequence-to-sequence language model fine-tuning to generate question-answer pairs given a context. |
| Outcome: | The proposed model outperforms other more convoluted approaches in the end-to-end model and is computationally light at both training and inference times. |
Scaling Laws for Linear Complexity Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing scaling laws for large language models are unclear, but they are useful for scalability. |
| Approach: | They propose scaling laws for linear complexity language models to establish a foundation for their scalability. |
| Outcome: | The proposed models demonstrate superior linguistic proficiency and knowledge retention. |
Multi-Level Information Retrieval Augmented Generation for Knowledge-based Visual Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge-Aware Visual Question Answering about Entity tasks require two separate steps to generate accurate answers. |
| Approach: | They propose a multi-level information RAG approach that enhances answer generation through entity retrieval and query expansion. |
| Outcome: | The proposed approach improves answer generation through entity retrieval and query expansion. |
TreeRare: Syntax Tree-Guided Retrieval and Reasoning for Knowledge-Intensive Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work shows that large language models generate incorrect statements due to over-reliance on parametric knowledge. |
| Approach: | They propose a framework that utilizes syntax trees to guide information retrieval and reasoning for question answering. |
| Outcome: | The proposed framework improves on existing state-of-the-art methods for large-scale query processing. |
AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark (2025.acl-long)
Copied to clipboard
| Challenge: | Evaluation benchmarks based on predefined domains and human-labeled data face limitations in addressing evaluation needs for emerging domains. |
| Approach: | They propose an automated information retrieval benchmark based on predefined domains and human-labeled data . AIR-Bench is automated and Heterogeneous with three key features . |
| Outcome: | The proposed benchmarks are based on predefined domains and human-labeled data. |
Modeling Complex Semantics Relation with Contrastively Fine-Tuned Relational Encoders (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning relational embeddings fail to capture nuanced representations and rich semantics. |
| Approach: | They propose different relational encoders designed to capture diverse relational aspects and semantic properties of entity pairs. |
| Outcome: | The proposed encoders capture diverse relational aspects and semantic properties of entity pairs. |
Search Wisely: Mitigating Sub-optimal Agentic Searches By Reducing Uncertainty (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have propelled their use in informationintensive tasks such as question answering and knowledge synthesis. |
| Approach: | They propose a reinforcement learning-based training method that incorporates confidence thresholds to reward high-certainty search decisions. |
| Outcome: | The proposed method outperforms baseline models on seven QA benchmarks and demonstrates that it is more efficient than existing methods. |
PairDistill: Pairwise Relevance Distillation for Dense Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in dense retrieval have demonstrated remarkable efficacy compared to traditional sparse retrieval methods. |
| Approach: | They propose to use pairwise relevance distillation to leverage pairwise reranking to enrich the training of dense retrieval models. |
| Outcome: | The proposed method outperforms existing methods and achieves state-of-the-art results on multiple benchmarks. |
Multi-Granularity Fusion Text Semantic Matching Based on WoBERT (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing text-matching methods struggle with semantic nuances in short texts . a novel approach to improve text semantic matching is being developed . |
| Approach: | They propose a multi-granularity fusion model that harnesses a pre-trained language model to capture text semantic nuances. |
| Outcome: | The proposed model improves on Chinese short text matching datasets compared to traditional methods . the proposed model captures individual text semantic nuances and improves accuracy . |
Link, Synthesize, Retrieve: Universal Document Linking for Zero-Shot Information Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for zero-shot learning are sparse, but have been used for dense retrieval (DR) . |
| Approach: | They propose a novel Universal Document Linking algorithm which links similar documents to enhance synthetic query generation across multiple datasets with different characteristics. |
| Outcome: | The proposed algorithm surpasses state-of-the-art methods in zero-shot cases. |
No Need for Large-Scale Search: Exploring Large Language Models in Complex Knowledge Base Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Knowledge Base Question Answering (KBQA) systems are a key research area in the field of natural language processing and information retrieval (IR). |
| Approach: | They propose to use large language models to convert natural language questions to structured knowledge representations by using a three-step fine-tune strategy to implement the KBQA system. |
| Outcome: | The proposed method achieves state-of-the-art performance across three datasets with a 79.9% F1 score. |
Stop Hardening Everything: A Training-Free Neuron-Level Defense for Neural Ranking Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses for neural ranking models are data-centric and require retraining and adversarial data generation. |
| Approach: | They propose a model-centric defense that addresses vulnerability at its architectural source without costly retraining or adversarial data generation. |
| Outcome: | The proposed approach outperforms state-of-the-art models on MS MARCO and TREC 19 while maintaining strong performance on clean data. |
Differentiated Vision: Unveiling Entity-Specific Visual Modality Requirements for Multimodal Knowledge Graph (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract features from images of entities overlook varying relevance of visual information across entities. |
| Approach: | a new model integrates structural and multimodal information of entities into a multimodal knowledge graph . a model evaluates the necessity of visual modality for each entity based on its attributes . |
| Outcome: | The proposed model improves on existing methods by adjusting visual data to different entity types. |
Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-conditioning has become the dominant approach for embedding specialization, but its direct application to low-capacity models imposes representational constraints that limit the performance gains derived from specialization. |
| Approach: | They propose a mixture of task experts transformer block which leverages task-specialized parameters trained with Task-Aware Contrastive Learning to enhance the model’s ability to generate specialized embeddings. |
| Outcome: | The proposed model achieves 64% higher performance gains in retrieval datasets (+3.27 +5.21) and 43% higher performance gain across all datasets (+1.81 2.60). |
Query-driven Relevant Paragraph Extraction from Legal Judgments (2024.lrec-main)
Copied to clipboard
| Challenge: | Legal professionals struggle with navigating lengthy legal judgements to pinpoint information that directly addresses their queries. |
| Approach: | They construct a specialized dataset to extract relevant paragraphs from legal judgements based on query . they assess the performance of current retrieval models in a zero-shot way . |
| Outcome: | The proposed model outperforms the current retrieval models in a zero-shot way and fine-tunes them using various models. |
InstructPart: Task-Oriented Part Segmentation with Instruction Reasoning (2025.acl-long)
Copied to clipboard
Zifu Wan, Yaqi Xie, Ce Zhang, Zhiqiu Lin, Zihan Wang, Simon Stepputtis, Deva Ramanan, Katia P. Sycara
| Challenge: | Large multimodal foundation models perceive objects as indivisible, overlooking the components that constitute them. |
| Approach: | They propose a novel benchmark for large multimodal foundation models comprising hand-labeled part segmentation annotations and task-oriented instructions to evaluate their performance. |
| Outcome: | The proposed benchmark improves performance of current models in understanding and executing part-level tasks within everyday contexts. |
RADCoT: Retrieval-Augmented Distillation to Specialization Models for Generating Chain-of-Thoughts in Query Expansion (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated superior performance to that of small language models in information retrieval for various subtasks including dense retrieval, reranking, query expansion, and pseudo-document generation. |
| Approach: | They propose a retrieval-augmented model specialization that distills the capability of LLMs to generate the chain-of-thoughts (CoT) for query expansion into a RADCoT. |
| Outcome: | The proposed model can generate the chain-of-thoughts (CoT) for query expansion, reducing the burden of internalizing and retaining world knowledge in model parameters. |
Spoken Document Retrieval for an Unwritten Language: A Case Study on Gormati (2025.findings-emnlp)
Copied to clipboard
Sanjay Booshanam, Kelly Chen, Ondrej Klejch, Thomas Reitmaier, Dani Kalarikalayil Raju, Electra Wallington, Nina Markl, Jennifer Pearson, Matt Jones, Simon Robinson, Peter Bell
| Challenge: | Speakers of unwritten languages have the potential to benefit from speech-based automatic information retrieval systems. |
| Approach: | They propose a speech embedding technique that facilitates a zero-shot speech-based automatic information retrieval system for unwritten languages. |
| Outcome: | The proposed method achieves a Top 5 retrieval rate of 87.9% on a corpus of Gormati, an unwritten language, that was collected in partnership with an agrarian Banjara community in Maharashtra State, India. |
Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to training with binary relevance labels uses synthetic data . contrastive learning with binary correlations leaves out subtle nuances useful for ranking . |
| Approach: | They propose to use waterstein distance as a loss function for training transformer-based retrievers with graduated relevance labels instead of real documents. |
| Outcome: | The proposed method outperforms conventional training with InfoNCE by a large margin on MARCO and BEIR benchmarks without using real documents. |
Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) suffer from inherent inabilities to interact with the physical world and access vast, up-to-date knowledge. |
| Approach: | They propose a tool retrieval benchmark for large language models (LLMs) that includes 7.6k diverse retrieval tasks and a corpus of 43k tools. |
| Outcome: | The proposed model performs poorly on the heterogeneous tool retrieval benchmark, resulting in low pass rate and low retrieval quality. |
GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs (2026.findings-acl)
Copied to clipboard
Meixiu Long, Duolin Sun, Dan Yang, Yihan Jiao, Lei Liu, Jiahai Wang, Binbin Hu, Yue Shen, Jie Feng, Zhehao Tan, Junjie Wang, Lianzhen Zhong, Jian Wang, Peng Wei, Jinjie Gu
| Challenge: | Existing rankers excel in lexical-matching scenarios, while they struggle with complex queries requiring deep reasoning. |
| Approach: | They propose a new paradigm that balances flexibility and context awareness to unlock the full potential of groupwise reranking. |
| Outcome: | The proposed approach achieves a state-of-the-art 65.2 NDCG@10 on BRIGHT and surpasses baselines by 2.1 points on R2MED while delivering a 6.4 inference speedup. |
CSTree-SRI: Introspection-Driven Cognitive Semantic Tree for Multi-Turn Question Answering over Extra-Long Contexts (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in natural language processing (NLP), particularly in single-turn question answering (QA) on short-text. |
| Approach: | They propose a framework that captures logical correlations across chunks of ELC and maintains coherence of multi-turn Questions. |
| Outcome: | The proposed framework is able to capture logical correlations across chunks of ELC and maintain coherence of multi-turn Questions. |
Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25 (2025.emnlp-main)
Copied to clipboard
| Challenge: | Interpretability in information retrieval (IR) models is coarse-grained and poorly understood . a cross-encoder model extracts traditional relevance signals, such as term frequency and inverse document frequency . |
| Approach: | They analyze how a common IR model extracts traditional relevance signals . this is similar to the probabilistic ranking function BM25 . |
| Outcome: | The proposed model extracts traditional relevance signals in early-to-middle layers, similar to BM25 . the model then combine these concepts in later layers, laying the groundwork for future interventions . |
Synergetic Interaction Network with Cross-task Attention for Joint Relational Triple Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to joint entity-relation extraction are limited in their ability to capture the interdependence between the two sub-tasks. |
| Approach: | They propose a synergistic approach to capture interdependence between named entity recognition and relation extraction sub-tasks in a Synergetic Interaction Network. |
| Outcome: | The proposed model achieves significantly better performance on three benchmark datasets. |
Embedding-Free RAG (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) is the current state-of-the-art method for mitigating the shortcomings of large language models. |
| Approach: | They propose a model-agnostic approach to retrieval-augmented generation that leverages generalized reasoning abilities of large language models. |
| Outcome: | Embedding-free RAG outperforms existing state-of-the-art methods in a wide range of domains. |
Beyond Markovian Forgetfulness: Episodic Memory for Reasoning-Intensive Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reasoning-intensive information retrieval suffer from inefficiency . Chain-of-Thought (CoT) approaches suffer from lack of token efficiency . Existing models lack episodic memory, which stores the history of prior states . |
| Approach: | They propose an algorithm that enhances state-based frameworks with an episodic memory module that stores the full history of prior states for a query. |
| Outcome: | The proposed model outperforms CoT and state-based models on the BRIGHT benchmark and is highly token-efficient. |
PL-MTEB: Polish Massive Text Embedding Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Text embeddings are used in many NLP tasks, including document clustering, semantic search, question answering, and classification. |
| Approach: | They introduce the Polish Massive Text Embedding Benchmark (PL-MTEB) it is a comprehensive benchmark for text embeddings in the Polish language. |
| Outcome: | The proposed model is based on 30 different NLP tasks in the Polish language. |
A Counterfactual Explanation Framework for Retrieval Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing literature on explainability of information retrieval has focused on illustrating the concept of relevance concerning a retrieval model. |
| Approach: | They propose to add terms to a document to improve its ranking to answer the question of which words played a role in not being favored by a retrieval model. |
| Outcome: | The proposed framework predicts counterfactuals for statistical and deep-learning models. |
SR-RAG: Verifiable Multi-Hop Reasoning via On-the-fly Symbolic Graph Construction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing paradigms for multi-hop reasoning suffer from high construction costs and limited adaptability to dynamic knowledge. |
| Approach: | They propose a symbolic reasoning framework for multi-hop question answering that integrates the advantages of both paradigms by dynamically generating sub-questions, performing information retrieval and symbolic encoding based on an on-the-fly graph and using a symbol verifier to validate intermediate reasoning steps. |
| Outcome: | The proposed framework significantly improves accuracy and robustness on multiple multi-hop benchmarks and a medical dataset. |
One Single Hub Text Breaks CLIP: Identifying Vulnerabilities in Cross-Modal Encoders via Hubness (2026.acl-long)
Copied to clipboard
| Challenge: | et al., 2010) show that hub embeddings are close to many unrelated examples in high-dimensional embeddable spaces . cross-modal encoders that project different modalities into a shared space are useful for cross-module applications . |
| Approach: | They propose a method for identifying the hub embedding and its corresponding hub text . they use images to evaluate cross-modal encoders that project different modalities into a shared space . |
| Outcome: | The proposed method can identify a single hub embedding and its corresponding hub text . it achieves comparable or higher similarity scores than human-written reference captions in many images . |